Augmented reality window

ABSTRACT

An apparatus comprises: a display device including an optically transparent display screen for displaying images to a user; a sensor configured to track a viewing direction and/or a viewing position of the user; a detector configured to determine a location of a target viewable by the user through the display screen; and a controller configured to control the display device to display an image on the display screen at screen coordinates corresponding with the viewable target, using the tracked viewing direction and/or the tracked viewing position of the user and the determined location of the target.

FIELD OF THE INVENTION

The present invention relates to augmented reality.

BACKGROUND

Generally, augmented reality (AR) relates to an interactive experienceof a real-world environment, in which objects in the real-worldenvironment are enhanced by computer-generated perceptual information,typically visual information, in real-time. AR may also be known asmixed reality or computer mediated reality. AR hence combines thecomputer-generated perceptual information with the real-worldenvironment.

Generally, a conventional AR device includes a processor and a memory, adisplay, a sensor for determining an orientation of the display andoptionally a camera. For example, smart phones and tablet computers maybe held (i.e. contact) and used as AR devices by users, in which livevideo images, acquired by the cameras and displayed on the displays, areoverlaid with computer-generated graphics. Head-mounted displays (HMDs),worn (i.e. contact) by users, are similar. AR eyeglasses, also worn byusers, may similarly display computer-generated graphics on eyeglassdisplays or by projection through or reflection off surfaces of theeyeglass lenses.

These conventional AR devices are individual (i.e. for a single user).Hence, a personal AR device is required for each user to experience theAR concurrently. Alternatively, a shared AR device is required for eachuser to experience the AR consecutively.

Hence, there is a need to improve AR devices.

SUMMARY OF THE INVENTION

It is one aim of the present invention, amongst others, to provide anapparatus and a method which at least partially obviates or mitigates atleast some of the disadvantages of the prior art, whether identifiedherein or elsewhere. For instance, it is an aim of embodiments of theinvention to provide an apparatus and a method that provides ARexperiences to users communally and/or contactlessly, without requiringindividual handheld or wearable AR devices, that overlays computerimagery for a specific tracked user on a large scale by utilising atransparent ‘window’ as said overlay. This operates in the sameprinciple as head mounted AR devices do, but requires the wearing of noequipment or physical contact with the user whatsoever in order tofunction. For instance, it is an aim of embodiments of the invention toprovide a method that indicates industrial robotic cell status throughthe projection of individual limb operational status onto thetransparent safety screen that surrounds it, tracking the viewer's eyelocation and respective robot location in order to achieve this method.

A first aspect provides an apparatus comprising:

-   -   a display device including an optically transparent display        screen for displaying images to a user;    -   a sensor configured to track a viewing direction and/or a        viewing position of the user;    -   a detector configured to determine a location of a target        viewable by the user through the display screen; and    -   a controller configured to control the display device to display        an image on the display screen at screen coordinates        corresponding with the viewable target, using the tracked        viewing direction and/or the tracked viewing position of the        user and the determined location of the target.

A second aspect provides a method of displaying an image to a user, themethod comprising:

-   -   tracking a viewing direction and/or a viewing position of the        user;    -   determining a location of a target viewable by the user through        an optically transparent display screen; and    -   displaying an image on the display screen at screen coordinates        corresponding with the viewable target, using the tracked        viewing direction and/or the tracked viewing position of the        user and the determined location of the target.

DETAILED DESCRIPTION OF THE INVENTION

According to the present invention there is provided an apparatus, asset forth in the appended claims. Also provided is a method. Otherfeatures of the invention will be apparent from the dependent claims,and the description that follows.

Apparatus

A first aspect provides an apparatus comprising:

-   -   a display device including an optically transparent display        screen for displaying images to a user;    -   a sensor configured to track a viewing direction and/or a        viewing position of the user;    -   a detector configured to determine a location of a target        viewable by the user through the display screen; and    -   a controller configured to control the display device to display        an image on the display screen at screen coordinates        corresponding with the viewable target, using the tracked        viewing direction and/or the tracked viewing position of the        user and the determined location of the target.

In this way, the user may view both the image, displayed on theoptically transparent display screen, and the target, through theoptically transparent display screen, thereby providing an AR experiencefor the user. Since the display screen is optically transparent, theuser may view the target directly therethrough, without requiringimaging of the target using a camera for example, thereby improvingviewing of the target and/or a field of view of the user. Since theviewing direction and/or the viewing position of the user are trackedand since the image is displayed at the screen coordinates based on thetracked viewing direction and/or the tracked viewing position of theuser, the screen coordinates of the image are updated or trackedaccordingly, for example as the user moves such that the tracked viewingdirection and/or the tracked viewing position of the user changes. Thatis, the image is displayed at screen coordinates that account forrelative movement between the user and the display device. Hence, theuser is not required to orient the display device so as to experiencethe AR, since the viewing direction and/or the viewing position of theuser is tracked. Similarly, the image is displayed at the screencoordinates based on the determined location of the target, which mayalso change. That is, the images displayed at screen coordinates thataccount also for relative movement between display device and thetarget. Hence, the image is displayed effectively superimposed upon thefield of view of the user. For example, the display device may be astatic display device, such as installed in a manufacturing facility, alaboratory, or a hospital, and controlled to display images to usersviewing targets through the optically transparent display screen.Particularly, the display device is neither wearable nor handheld, notrequiring wearing, holding or contact by users. Hence, personal ARdevices are not required for each user to experience the AR concurrentlywhile sharing of a shared device is not required for each user toexperience the AR consecutively. That is, the number of AR devices isreduced and wearing or holding of AR devices eliminated while sharing,requiring contact and potentially cross contamination, of a single ARdevice is avoided.

It should be understood that smartphones and tablets do not haveoptically transparent display screens, as described below, since targetsare not viewable by users therethrough. Instead, live video images,acquired by the cameras and displayed on the displays of the smartphonesand tablets, are overlaid with computer-generated graphics. In oneexample, the display device is not a smartphone or a tablet.

It should be understood that HMDs and AR eyeglasses may have opticallytransparent display screens. However, such a wearable AR devicetypically determines a viewing direction and/or a viewing position ofthe AR device, rather than of the user, as described below.

The apparatus comprises the display device, the sensor, the detector andthe controller. It should be understood that the controller iscommunicatively coupled, unidirectionally or bidirectionally asappropriate and wired and/or wirelessly, to the display device, thesensor and the detector. In one example, the display device, the sensor,the detector and/or the controller are physically separate, for examplein separate housings, and may be spaced apart. In one example, thedisplay device, the sensor, the detector and/or the controller areintegrated, for example in the same housing.

Display Device

The apparatus comprises the display device including the opticallytransparent display screen for displaying the images to the user.

In one example, the display screen comprises and/or is a see-throughdisplay. A see-through display is a term of the art and is an electronicdisplay that allows the user to see what is shown on the opticallytransparent display screen while still being able to see therethrough.See-through displays are known and include LCD and LED (including bothinorganic and organic LED such as transparent organic light emittingdevice, TOLED) display screens. Passive transparent displays are alsoknown. Other see-through displays are known. Such a see-through displaymay have a transmittance of about 70%.

In one example, the display screen comprises and/or is a projectionscreen, for example a front projection screen and/or a rear projectionscreen) and wherein the display device comprises a projector arranged toproject the image on the display screen.

In one example, the projection screen comprises and/or is asemi-transparent projection screen having a rear or front projectionfilm or coating or an optical diffusion screen. Such projection screensare typically used for holographic or ‘Pepper's Ghost’ projection. Sucha projection screen may have a transmittance of 80% or more. Suitableprojectors are known.

In one example, the projection screen comprises and/or is a beamsplitter. In this way, a head-up display (HUD) may be provided. Atypical HUD contains three primary components: a projector unit, acombiner, and a video generation computer. The projection unit in atypical HUD is an optical collimator setup: a convex lens or concavemirror with a cathode ray tube, light emitting diode display, or liquidcrystal display at its focus, thereby displaying the image where thelight is collimated i.e. the focal point is perceived to be at infinity.The combiner is typically an angled flat piece of glass (a beamsplitter) located directly in front of the user, that redirects theprojected image from projector in such a way that the user may view thefield of view and the projected infinity image at the same time.Combiners may have coatings that reflect the monochromatic lightprojected onto it from the projector unit while allowing all otherwavelengths of light to pass through. In some optical layouts, combinersmay also have a curved surface to refocus the image from the projector.

In one example, the display screen has a dimension, for example adiagonal dimension, in a range from 0.5 m to 25 m, preferably in a rangefrom 1 m to 10 m, more preferably in a range from 2 m to 5 m. That is,the display screen is relatively large, compared with a display screenof a handheld or wearable AR device.

In one example, the display screen is adapted to provide a safetyscreen, disposed between the user and the target. In this way, thedisplay screen isolates the user from the target, by providing a barriertherebetween, formed thereby, preventing the user from contacting thetarget and/or vice versa.

In one example, the display screen encloses, at least in part, thetarget. For example, the display screen may surround the target on oneor more sides, such as encircling the target.

In one example, the display device comprises and/or is a stereo displaydevice (also known as a 3D display device), for example a stereo displaydevice requiring glasses or an autostereoscopic display not requiringglasses, as understood by the skilled person. In this way, the image maybe displayed at predetermined depth and/or at a depth corresponding withthe viewable target, for example a similar depth. In this way, viewingof the displayed by the user is facilitated, since the user may focus onthe target and the image, for example simultaneously.

Sensor

The apparatus comprises the sensor configured to track the viewingdirection and/or the viewing position of the user. It should beunderstood that the sensor is configured to track the viewing directionand/or the viewing position of the user, for example the gaze or line ofsight thereof. In contrast, handheld and wearable AR devices areconfigured to track the viewing direction and/or the viewing position ofthese AR devices. That is, a user of a handheld AR device may beorienting the handheld AR device in a particular direction but viewingin a different direction. Similarly, a user of a wearable AR device maybe orienting the wearable AR device in a particular direction but gazingin a different direction. Hence, the apparatus according to the firstaspect is responsive to the current viewing direction and/or the currentviewing position of the user. In this way, the image displayed by on thedisplay screen is responsive to, for example at screen coordinatesand/or generated according to, where the user is looking. In oneexample, the sensor is configured to track the gaze, the direct vision,the central vision, the para-central vision, the macula vision and/orthe line of sight of the user. In one example, the sensor is configuredto track the indirect vision (also known as the peripheral vision), theperipheral vision, the mid-peripheral vision and/or the far peripheralvision of the user.

It should be understood that the sensor is configured to track, forexample currently, intermittently or continuously, the viewing directionand/or the viewing position of the user. In this way, the screencoordinates of the image and/or the image may be updated according tothe tracked viewing direction and/or tracked viewing position, so as tofollow where the user is looking and/or movement of the user. In oneexample, the sensor is configured to track the viewing direction and/orthe viewing position of the user by sensing a position of the user'seyes and/or pupils and/or the respective positions of the user's eyesand/or pupils, for example using facial capture, direction of gazedetection and/or eye location software applied to images, for examplecamera or video images, of the user. Such software is known. Eyelocation is preferred, requiring a relatively simple camera and havingrelatively low computer resource requirements compatible with real-timeprocessing, while allowing relatively greater freedom of movement of theuser. In contrast, facial capture and/or direction of gaze detection mayrequire relatively complex cameras, suitable lighting and relativelyhigh computer resource requirements.

It should be understood that the sensor is configured to track theviewing direction and/or the viewing position of the user at aresolution, a precision and/or with an accuracy sufficient to calculatescreen coordinates for the image, as described below.

In one example, the sensor comprises and/or is a camera, anaccelerometer, a gyroscope, a solid state compass, a radio-frequencyidentification (RFID) detector and/or a positional sensor, for example aglobal positioning system, GPS. Preferably, the sensor comprises and/oris a camera and suitable software. Suitable sensors include a Microsoft®Azure® Kinect®, VicoVR® available from https://vicovr.com/, OrbbecAstra® available from https://orbbec3d.com/, Stereolabs ZED® availablefrom https://www.stereolabs.com/ and OpenPose® available fromhttps://github.com/CMU-Perceptual-Computing-Lab/openpose. Other sensorsare known.

In one example, the sensor is configured to track respective viewingdirections and/or the respective viewing positions of a plurality ofusers.

Detector

The apparatus comprises the detector configured to determine thelocation of the target viewable by the user through the display screen.

It should be understood that the target is viewable by the user throughthe display screen since the display screen is optically transparent.That is, the target is not an image displayed on the display screen butrather, an object on the other side of the display screen with respectto the user. It should be understood that the location of the targetcomprises and/or is a two-dimensional or a three-dimensional location,that may be absolute or relative to the display screen and/or the user,such as a geographic location, optionally including elevation, orlocation relative to a coordinate system. It should be understood thatthe detector is configured to determine the location of the target at aresolution, a precision and/or with an accuracy sufficient to calculatescreen coordinates for the image, as described below.

The detector may be as described with respect to the sensor.

In one example, the detector is configured to track the location of thetarget (and/or more parts thereof), as described with respect totracking the viewing direction and/or the viewing position of the userby the sensor, mutatis mutandis. Hence, the apparatus according to thefirst aspect may be responsive to the tracked location of the target. Inthis way, the image displayed on the display screen may be responsiveto, for example at screen coordinates and/or generated according to, thecurrent location of the target.

In one example, the detector comprises and/or is a camera, anaccelerometer, a gyroscope, a solid state compass, a radio-frequencyidentification (RFID) detector, an Ultra-Wideband, UWB, sensor and/or apositional sensor, for example a global positioning system, GPS.

In one example, the detector is communicatively coupleable, for exampleunidirectionally or bidirectionally and wired or wirelessly, to thetarget and wherein the detector is configured to determine the locationof the target (and/or one or more parts thereof) based on position datareceived therefrom. That is, the detector may comprise a software sensor(also known as a soft sensor all virtual sensor), configured to read,infer and/or calculate the location of the target (and/or one or moreparts thereof) using the position data received therefrom. For example,if the target is an industrial robot or part thereof, the industrialrobot may output digital readout (DRO) data including locations of axes.

In one example, the detector is configured to determine respectivelocations of a plurality of targets (and/or one or more parts thereof)viewable by the user through the display screen. Preferably, saidlocations are communicated from the said target's (or targets')respective controller to the detector. In other words, direct data feedsin real time from the target(s)′ controller on the location of thetarget(s) (and/or one or more parts thereof) are communicated to thedetector. In this way, the present invention advantageously continuouslymonitors the complex positioning and movement of the target(s) (and/orone or more parts thereof) behind the display device. Preferably,streaming data direct from the target(s)′ controller in real time isutilised to ascertain and continually monitor such positions. Thisfunctionality is particularly advantageous with regard to complextargets with multiple parts and/or multiple targets.

Target

In one example, the target comprises and/or is industrial, agriculturalor medical machinery and/or one or more parts of thereof. Examples ofindustrial machinery include manufacturing, handling and assemblinglines for engineering and construction and chemical process plants.Examples of agricultural machinery include farming equipment andprocessing. Examples of medical machinery include diagnostic, surgicaland therapeutic equipment. In one example, the target is an industrialrobot and/or one or more parts thereof, for example an axis (or axes)thereof. In one example, the target comprises and/or is a land craft,watercraft and/or aircraft, for example for road traffic or air trafficmanagement, such as at a bus station, train station or airport. Forexample, passengers (i.e. users) may view buses (i.e. targets) throughthe display device at a bus station, which provides a barriertherebetween, and the display device may display associated bustimetables (i.e. images) thereon. In one example, the target comprisesand/or is animal, including a human, for security monitoring, sport orhospitality, for example. For example, servers (i.e. users) may viewtables and diners (i.e. targets) through the display device at arestaurant, which provides a barrier therebetween, and the displaydevice may display associated orders (i.e. images) thereon. In this way,the user may view the target T (T1 to T5) through the display screen,together with the displayed image.

In one example, the target comprises and/or is an industrial robot or apart thereof. In a preferred example the target(s) is a robot.

In one example, the apparatus comprises the target.

Preferably the target comprises one or more parts thereof, morepreferably the target comprises multiple parts thereof. Preferably, thetarget is a multi-axis robot.

Controller

The apparatus comprises the controller configured to control the displaydevice to display the image on the display screen at the screencoordinates corresponding with the viewable target, using the trackedviewing direction and/or the tracked viewing position of the user andthe determined location of the target. It should be understood that thecontroller is configured to calculate the screen coordinates using thetracked viewing direction and/or the tracked viewing position of theuser and the determined location of the target, for example by geometricconstruction. Such geometric construction is known. Other methods ofcalculating the screen coordinates are known. In one example, thecontroller is configured to control the display device to repeatedly,for example intermittently, periodically or continuously, calculate thescreen coordinates. In this way, the screen coordinates of the image maybe updated according to relative movement of the user and/or the targetwith respect to the display screen, such that the image is displayed onthe display screen at a substantially constant placement relative to thetarget and the user. That is, as the user and/or the target moves, theimage is displayed at updated screen coordinates that account for thismovement. It should be understood that the images displayed at thescreen coordinates corresponding with the viewable target, for exampleoverlaying the viewable target, adjacent thereto or at a predeterminedscreen coordinates, for example as viewed by the user having the trackedviewing direction and/or the tracked viewing position.

In one example, the controller is configured to control the displaydevice to display a plurality of images on the display screen atrespective screen coordinates corresponding with a plurality of viewabletargets and/or one or more parts thereof, using the tracked viewingdirection and/or the tracked viewing position of the user and therespective determined locations of the plurality of targets (and/or oneor more parts thereof).

In one example, the controller comprises a processor and a memory, forexample a computer.

Image

It should be understood that generally, the image is associated with thetarget, for example computer-generated visual information about thetarget. In one example, the image comprises and/or is text and/or agraphic, such as an icon, a shape, a picture, a model. In one example,the image is a static graphic. In one example, the image is a dynamicgraphic.

Identifier

In one example, the apparatus comprises an identifier configured toestablish an identity of the user and the controller is configured tocontrol the display device to display the image based on the establishedidentity. In this way, display of the image may be conditional (i.e.displayed or not displayed) according to the established identity and/orthe computer-generated visual information about the target may bespecific for the user or a user group thereof. It should be understoodthat the identifier is thus a sensing element configured to identify theuser. For example, the identifier may establish the identity of the userusing facial recognition from an image of the user acquired by a camera.For example, the identifier may establish the identity of the user byreading an RFID tag of the user. Generally, the identifier may be asdescribed with respect to the sensor, mutatis mutandis.

User Input Device

In one example, the apparatus comprises a user input device configuredto receive an input from the user. In this way, the user may interactwith the apparatus.

In one example, the user input device comprises a gesture sensor and/orthe display screen comprises a touch screen. In this way, the user mayinteract with the apparatus using gestures (i.e. non-contact) and/or viatouch (i.e. contact or near contact).

In one example, the controller is communicatively coupleable to thetarget and the controller is configured to control the target accordingto the input received from the user. In this way, the user may controlthe target remotely, via the apparatus, for example by gesture and/ortouch. For example, the user may view the target through the displayscreen together with the image displayed thereon and control the target,whilst viewing the target through the display screen together with theimage displayed thereon.

Method

The second aspect provides a method of displaying an image to a user,the method comprising:

-   -   tracking a viewing direction and/or a viewing position of the        user;    -   determining a location of a target viewable by the user through        an optically transparent display screen; and    -   displaying an image on the display screen at screen coordinates        corresponding with the viewable target, using the tracked        viewing direction and/or the tracked viewing position of the        user and the determined location of the target.

The tracking, the viewing direction, the viewing position, the user, thedetermining, the location, the target, the optically transparent displayscreen, the image and/or the screen coordinates maybe as described inrespect of the first aspect.

Definitions

Throughout this specification, the term “comprising” or “comprises”means including the component(s) specified but not to the exclusion ofthe presence of other components. The term “consisting essentially of”or “consists essentially of” means including the components specifiedbut excluding other components except for materials present asimpurities, unavoidable materials present as a result of processes usedto provide the components, and components added for a purpose other thanachieving the technical effect of the invention, such as colourants, andthe like.

The term “consisting of” or “consists of” means including the componentsspecified but excluding other components.

Whenever appropriate, depending upon the context, the use of the term“comprises” or “comprising” may also be taken to include the meaning“consists essentially of” or “consisting essentially of”, and also mayalso be taken to include the meaning “consists of” or “consisting of”.

The optional features set out herein may be used either individually orin combination with each other where appropriate and particularly in thecombinations as set out in the accompanying claims. The optionalfeatures for each aspect or exemplary embodiment of the invention, asset out herein are also applicable to all other aspects or exemplaryembodiments of the invention, where appropriate. In other words, theskilled person reading this specification should consider the optionalfeatures for each aspect or exemplary embodiment of the invention asinterchangeable and combinable between different aspects and exemplaryembodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention, and to show how exemplaryembodiments of the same may be brought into effect, reference will bemade, by way of example only, to the accompanying diagrammatic Figures,in which:

FIG. 1 schematically depicts an apparatus according to an exemplaryembodiment;

FIG. 2 schematically depicts a method according to an exemplaryembodiment;

FIG. 3A schematically depicts a plan perspective view of an apparatusaccording to an exemplary embodiment; and

FIG. 3B schematically depicts a front elevation perspective view of theapparatus.

DETAILED DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically depicts an apparatus 10 according to an exemplaryembodiment.

The apparatus 10 comprises:

-   -   a display device 110 including an optically transparent display        screen 111 for displaying images to a user U;    -   a sensor 120 configured to track a viewing direction VD and/or a        viewing position VP of the user U;    -   a detector 130 configured to determine a location L of a target        T viewable by the user U through the display screen 111; and    -   a controller 140 configured to control the display device 110 to        display an image 150 on the display screen 111 at screen        coordinates corresponding with the viewable target T, using the        tracked viewing direction VD and/or the tracked viewing position        VP of the user U and the determined location L of the target T.

FIG. 2 schematically depicts a method according to an exemplaryembodiment.

The method is of displaying an image to a user.

At S201, the method comprises tracking a viewing direction and/or aviewing position of the user.

At S202, the method comprises determining a location of a targetviewable by the user through an optically transparent display screen.

At S203, the method comprises displaying an image on the display screenat screen coordinates corresponding with the viewable target, using thetracked viewing direction and/or the tracked viewing position of theuser and the determined location of the target.

FIG. 3A schematically depicts a plan perspective view of an apparatusaccording to an exemplary embodiment; and FIG. 3B schematically depictsa front elevation perspective view of the apparatus 20.

The apparatus 20 comprises:

-   -   a display device 210 including an optically transparent display        screen 211 for displaying images to a user U;    -   a sensor 220 configured to track a viewing direction VD and/or a        viewing position VP of the user U;    -   a detector 230 (not shown) configured to determine a location L        (L1 to L5) of a target T (T1 to T5) viewable by the user U        through the display screen 211; and    -   a controller 240 (not shown) configured to control the display        device 210 to display an image 250 (250A to 250H) on the display        screen 211 at screen coordinates corresponding with the viewable        target T (T1 to T5), using the tracked viewing direction VD        and/or the tracked viewing position VP of the user U and the        determined location L (L1 to L5) of the target T (T1 to T5).

Briefly, FIGS. 3A and 3B show overlaid computer images for a specifictracked user on a large scale by utilising a transparent ‘window’ forsaid overlay. This operates the same AR principle as head mounted ARdevices, but requires the wearing of no equipment or physical contact bythe user whatsoever in order to function. In this example, the imagesindicate industrial robotic cell status through the projection ofindividual limb (axis) operational status onto the transparent safetyscreen that surrounds it, tracking the user's eye location andrespective robot location in order to achieve this method.

In this example, the display device 210, the sensor 220, the detector240 and/or the controller 250 are physically separate, for example inseparate housings, and are spaced apart.

In this example, the display screen 211 is a projection screen 211,particularly a rear projection screen 211 and wherein the display device210 comprises a projector 212 arranged to project the image 250 (250A to250H) on the display screen 211. In this example, the projection screen211 comprises and/or is a semi-transparent projection screen 211 havinga rear projection film. In this example, the display screen 211 has adiagonal dimension of 1.5 m. In this example, the display screen 211 isadapted to provide a safety screen 211, disposed between the user U andthe target T (T1 to T5).

In this example, the display screen 211 encloses, at least in part, thetarget T (T1 to T5).

In this example, the sensor 220 is configured to track the viewingdirection VD and/or the viewing position VP of the user U by sensing aposition of the user U's eyes and/or pupils and/or the respectivepositions of the user U's eyes and/or pupils, using eye location L (L1to L5) software applied to images, for example camera or video images,of the user U. In this example, the sensor 220 comprises a camera andsuitable software. In this example, the sensor 220 is configured totrack respective viewing direction VDs and/or the respective viewingposition VPs of a plurality of users.

In this example, the detector 240 is communicatively coupled,bidirectionally and wired, to the target T (T1 to T5) and wherein thedetector 240 is configured to determine the location L (L1 to L5) of thetarget T (T1 to T5) based on position data received therefrom. In thisexample, the target T (T1 to T5) is an industrial robot or part thereof,the industrial robot may output digital readout (DRO) data includinglocations L (L1 to L5) of axes. In this example, the detector 240 isconfigured to determine respective locations of a plurality of targetsT1 to T5 viewable by the user U through the display screen 211. In thisexample, the target T (T1 to T5) comprises and/or is an industrial robotor a part thereof.

In this example, the controller 250 is configured to control the displaydevice 210 to repeatedly, for example intermittently, periodically orcontinuously, calculate the screen coordinates. In this example, thecontroller 250 is configured to control the display device 210 todisplay a plurality of images 250 (250A to 250H) on the display screen211 at respective screen 211 coordinates corresponding with a pluralityof viewable targets T (T1 to T5), using the tracked viewing direction VDand/or the tracked viewing position VP of the user U and the respectivedetermined locations L (L1 to L5) of the plurality of targets T (T1 toT5). In this example, the controller 250 comprises a processor and amemory, for example a computer.

It should be understood that generally, the image 250 (250A to 250H) isassociated with the target T (T1 to T5), for example computer-generatedvisual information about the target T (T1 to T5). In this example, theimage 250 (250A to 250H) comprises and/or is text and/or a graphic.

Although a preferred embodiment has been shown and described, it will beappreciated by those skilled in the art that various changes andmodifications might be made without departing from the scope of theinvention, as defined in the appended claims and as described above.

Attention is directed to all papers and documents which are filedconcurrently with or previous to this specification in connection withthis application and which are open to public inspection with thisspecification, and the contents of all such papers and documents areincorporated herein by reference.

All of the features disclosed in this specification (including anyaccompanying claims and drawings), and/or all of the steps of any methodor process so disclosed, may be combined in any combination, exceptcombinations where at most some of such features and/or steps aremutually exclusive.

Each feature disclosed in this specification (including any accompanyingclaims, and drawings) may be replaced by alternative features servingthe same, equivalent or similar purpose, unless expressly statedotherwise. Thus, unless expressly stated otherwise, each featuredisclosed is one example only of a generic series of equivalent orsimilar features.

The invention is not restricted to the details of the foregoingembodiment(s). The invention extends to any novel one, or any novelcombination, of the features disclosed in this specification (includingany accompanying claims and drawings), or to any novel one, or any novelcombination, of the steps of any method or process so disclosed.

1: An apparatus comprising: a display device including an opticallytransparent display screen for displaying images to a user; a sensorconfigured to track a viewing direction and/or a viewing position of theuser; a detector configured to determine a location of a target viewableby the user through the display screen; and a controller configured tocontrol the display device to display an image on the display screen atscreen coordinates corresponding with the viewable target, using thetracked viewing direction and/or the tracked viewing position of theuser and the determined location of the target. 2: The apparatusaccording to claim 1, wherein the controller is configured to controlthe display device to display a plurality of images on the displayscreen at respective screen coordinates corresponding with a pluralityof viewable targets, using the tracked viewing direction and/or thetracked viewing position of the user and the respective determinedlocations of the plurality of targets. 3: The apparatus according toclaim 1, wherein the display screen comprises a projection screen andwherein the display device comprises a projector arranged to project theimage on the display screen. 4: The apparatus according to claim 1,wherein the display screen is adapted to provide a safety screen,disposed between the user and the target. 5: The apparatus according toclaim 4, wherein the display screen encloses, at least in part, thetarget. 6: The apparatus according to claim 1, wherein the sensorcomprises a camera, an accelerometer, a gyroscope, a solid statecompass, a radio-frequency identification detector, and/or a positionalsensor. 7: The apparatus according to claim 1, wherein the detectorcomprises a camera, an accelerometer, a gyroscope, a solid statecompass, a radio-frequency identification detector, an Ultra-Wideband,UWB, sensor, and/or a positional sensor. 8: The apparatus according toclaim 1, wherein the detector is communicatively coupleable to thetarget and wherein the detector is configured to determine the locationof the target based on position data received therefrom. 9: Theapparatus according to claim 1, comprising an identifier configured toestablish an identity of the user and wherein the controller isconfigured to control the display device to display the image based onthe established identity. 10: The apparatus according to claim 1,comprising a user input device configured to receive an input from theuser. 11: The apparatus according to claim 10, wherein the user inputdevice comprises a gesture sensor and/or wherein the display screencomprises a touch screen. 12: The apparatus according to claim 10,wherein the controller is communicatively coupleable to the target andwherein the controller is configured to control the target according tothe input received from the user. 13: The apparatus according to claim1, wherein the apparatus comprises the target. 14: The apparatusaccording to claim 13, wherein the target comprises an industrial robot.15: A method of displaying an image to a user, the method comprising:tracking a viewing direction and/or a viewing position of the user;determining a location of a target viewable by the user through anoptically transparent display screen; and displaying an image on thedisplay screen at screen coordinates corresponding with the viewabletarget, using the tracked viewing direction and/or the tracked viewingposition of the user and the determined location of the target.